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Background Loss to follow-up of trial participants represents a threat to research 
validity. To date, interventions designed to increase participants' awareness of 
benefits to society of completing follow-up, and the impact of a telephone call from 
a senior female clinician and researcher requesting follow-up have not been 
evaluated robustly. 

Purpose Trial 1 aimed to evaluate the effect on trial follow-up of written information 
regarding the benefits of participation to society. Trial 2 aimed to evaluate the effect 
on trial follow-up of a telephone call from a senior female clinician and researcher. 
Methods Two single-blind randomized controlled trials were nested within a larger 
trial, Txt2stop. In Trial 1, participants were allocated using minimization to receive a 
refrigerator magnet and a text message emphasizing the benefits to society of 
completing follow-up, or to a control group receiving a simple reminder regarding 
follow-up. In Trial 2, participants were randomly allocated to receive a telephone 
call from a senior female clinician and researcher, or to a control group receiving 
standard Txt2stop follow-up procedures. 

Results Trial 1 : 33.5% (327 of 976) of the intervention group and 33.8% (329 of 
974) of the control group returned the questionnaire within 26 weeks of random- 
ization, risk ratio (RR) 0.99; 95% confidence interval (CI) 0.88-1.12. In all, 83.3% 
(81 3 of 976) of the intervention group and 82.2% (801 of/974) of the control group 
sent back the questionnaire within 30 weeks of randomization, RR 1 .01 ; 95% CI 0.97, 
1 .05. Trial 2: 31 % (20 of 65) of the intervention group and 32% (20 of 62) of the 
control group completed trial follow-up, RR 0.93; 95%CI 0.44, 1 .98. 
Conclusions In presence of other methods to increase follow-up neither experi- 
mental method (refrigerator magnet and text message emphasizing participation's 
benefits to society nor a telephone call from study's principal investigator) increased 
participant follow-up in the Txt2stop trial. Clinical Trials 201 1 ; 8: 654-660. http:// 
ctj.sagepub.com 



Introduction 

Randomized controlled trials (RCTs) generally are 
considered the gold standard in evaluation of 



health care interventions but losses to follow-up 
represent an important threat to both their internal 
and external validity [1-3] . Bias may occur as those 
lost to follow-up are likely to differ from those for 



a Health Protection Agency, London, UK, b Nutrition and Public Health Intervention Research Unit, London School of 
Hygiene and Tropical Medicine, London, UK 

Author for correspondence: Ettore Severi, Health Protection Agency, South East Regional Epidemiology Unit, 
151 Buckingham Palace Road, London SW1W 9SZ, UK 
Email: ettore.severi@hpa.org.uk 



© The Author(s), 201 1 . 

Reprints and permissions: http://www.sagepub.co.uk/journalsPermissions.nav 10.1177/1740774511416524 



Mobile phone-based smoking cessation support 655 



whom follow-up is completed and bias may be 
more pronounced where there is differential follow- 
up between the intervention and control groups 
[4]. Sprague et al. [4] suggest that under 5% loss to 
follow-up will result in little bias, while a loss 
greater than 20% may represent a serious threat to 
the validity of the study. However, as the amount 
of bias introduced depends on the differences 
between those who were and were not followed- 
up and is trial specific, the level of bias caused by 
losses to follow-up cannot be accurately deter- 
mined [4], The potential for bias introduced by 
losses to follow-up can only be completely elimi- 
nated by achieving 100% follow-up, so it is imper- 
ative that trials aim to achieve as close to complete 
follow-up as possible [5]. 

Many trials fail to achieve high follow-up, 
potentially wasting economic and intellectual 
resources. A review of participant recruitment and 
retention in RCTs in six major journals from 2009 
showed that 48% of trials reporting a sample size 
calculation failed to achieve adequate numbers at 
outcome assessment, once those lost to follow-up 
were excluded [6]. Economic and intellectual 
resources allocated to research studies are not 
limitless, and study validity compromised by exces- 
sive loss to follow-up is a waste of valuable 
resources. 

A wide range of methods to increase response 
rates for postal, e-mail, and telephone question- 
naires have been evaluated in RCTs, including the 
length of questionnaire, prenotification, number of 
requests, the nature and style of questions, incen- 
tives, status of sender, and method of delivery [7] . 

The effects on follow-up of interventions that 
emphasize trial participation's benefits to society 
are uncertain [7]. In all, 10 trials (with a total of 
12,731 participants) evaluated the effect of an 
appeal stressing the benefit to society if participants 
return a questionnaire: pooled odds ratio (OR) 1.09 
(95% confidence interval [CI] 0.92 to 1.29) [7]. The 
trials, however, are affected by several design 
weaknesses: none of the 10 trials gave evidence of 
the methods of randomization, except one [8], 
where allocation was not through randomization 
but systematic; there was no evidence that alloca- 
tion of study arm was concealed from trial staff; and 
all the trials showed evidence of selection bias [7] . 

The effect of a telephone reminder from a senior 
female clinician/researcher on response rates is not 
clear. Trials sometimes use a senior staff member to 
contact non-respondents with the aim of reducing 
loss to follow-up. Completed trials showed no 
evidence of effect, when a senior or well-known 
person signed letters accompanying questionnaires: 
pooled ORs 1.05 (95% CI 0.89 to 1.23) and OR 1.05 
(95% CI 0.95 to 1.15), respectively [7]. There was 
also no evidence of effect on response for the 



gender of the person requesting follow-up ques- 
tionnaires (OR 1.07; 95% CI 0.72 to 1.58) but in one 
trial the odds of response decreased by over a half, 
when the electronic questionnaire was signed by a 
man (OR 0.55; 95% CI 0.38 to 0.80) [7]. It is 
plausible that personal contact by telephone may 
influence response rates, especially in a clinical trial 
where the call comes from a senior female clinician 
and researcher, but to date there have been no trials 
evaluating this approach. 

Txt2stop is a RCT with 5800 participants that has 
been designed to evaluate the effect of mobile 
phone-based smoking cessation support on smok- 
ing rates at 6 months after enrolment. The eligibil- 
ity criteria for Txt2stop were: aged 16 years or over; 
daily smoker; willing to quit in the next month; 
owned a mobile phone and resident in the United 
Kingdom. The primary outcome of Txt2stop was 
biochemically verified continuous abstinence at 
6 months. Data regarding self-reported smoking 
status were collected and, from those reporting 
continuous abstinence, a salivary sample for cotin- 
ine level assessment was requested by post. Previous 
trials of smoking cessation support, particularly 
those using new technologies to deliver support, 
have experienced high and differential losses to 
follow-up for long-term outcomes [9,10]. One of 
our aims in the Txt2stop trial was to minimize loss 
to follow-up. We followed-up participants by any 
of the means they agreed to at the start of the trial, 
including post, e-mail, and telephone calls to 
mobile, home, or work numbers [11]. We used all 
the effective evidence-based methods that were 
feasible to introduce into the procedures of the trial 
[12], as identified in the systematic reviews by 
Edwards et al. and Hoile et al. [12,13]. These 
included monetary incentives, posting correspon- 
dence by recorded delivery, pre-notification, 
follow-up contact, unconditional advance cash 
incentives, short, concise questionnaires, duplicate 
questionnaires sent at repeat follow-up attempts, 
mentioning that commitment to the trial implied 
an obligation to respond, mention of university 
sponsorship, prepaid return envelopes with stamps, 
an assurance of confidentiality, and first-class out- 
ward mailing. 

In addition to these procedures, we generated 
two hypotheses to test in two trials conducted 
among Txt2stop participants. 

The first research hypothesis was that telling 
study participants that their participation in 
research could benefit society, would reduce loss 
to follow-up in the Txt2stop trial. 

The second research hypothesis was that a 
telephone call from the study principal investiga- 
tor (female senior clinician and researcher) explain- 
ing the importance of follow-up and asking 
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participants to complete their participation would 
reduce loss to follow-up in the Txt2stop trial. 



Methods 

We obtained ethical approval for these trials from 
the London School of Hygiene and Tropical 
Medicine Ethics Committee. 

Study design and characteristics of the two trials 
are described in Table 1. 

Trial 1 was a RCT evaluating the impact of an 
intervention, that is, providing information regard- 
ing the benefits to society of participation, on 
participants' follow-up. This was a single-blind 
controlled trial, with those recording and assessing 
outcomes blind to the intervention. 

In addition to the standard Txt2stop follow-up 
procedures, the intervention group was sent writ- 
ten information on a refrigerator magnet by post 
(see Figure 1), between 16 and 20 weeks after 
randomization into the Txt2stop trial, followed by 
a mobile phone text message 3 days after the 
Txt2stop postal follow-up questionnaire was sent. 
We aimed to sensitize participants by making them 
aware of the social benefits of remaining in the 
study for its full duration, regardless of smoking 
cessation status. 

The initial sensitization consisted of a message 
on a refrigerator magnet within a sealed envelope. 
The message said that medical research is 



important to society and pointed out that by 
taking part in Txt2stop, the participant was bene- 
fiting society. 

The text message said 'Be proud of yourself for 
helping medical research! Thank you for filling in 
the Txt2stop questionnaire.' The control group 
received a text message reminding the participant 
the follow-up questionnaire was due 3 days after 
the Txt2stop postal questionnaire had been sent 
(Week 23). The text message said 'Thank you for 
filling in the Txt2stop questionnaire.' 

The 1950 Txt2stop participants who enrolled 
between 1 March 2009 and 1 June 2009 and had 
provided postal addresses were eligible for Trial 1 
[12]. They were able to withdraw at any time by 
texting 'stop' to the short code 65151. Any partic- 
ipant withdrawing from the Txt2stop trial was also 
withdrawn from Trial 1. 

The participants were allocated to Trial 1 inter- 
vention or control through minimization (using 
Minim software [14] to balance four different 
characteristics affecting participant retention in 
Txt2stop: age, sex, Fagerstrom index (measure of 
nicotine dependency), and allocation to interven- 
tion or control group in Txt2stop. The allocation of 
the participants to Trial 1 intervention or control 
group was concealed from the investigators. 

The primary outcome of the study was com- 
pleted follow-up questionnaires at 30 weeks from 
randomization. The secondary outcome was com- 
pleted follow-up questionnaires at 26 weeks from 
randomization. 



Table 1 Studies design and characteristics 



Trial 1 



Trial 2 



Study design 
Intervention 



Control 

Eligibility criteria 

Trial consent 

Withdraw 
Allocation method 
Primary outcome 

Secondary outcome 
Statistical analysis 



Sample size 



Single-blind randomized control trial 
Refrigerator magnet by post and telephone text 

message emphasizing social benefits of study 

participation 
Text message reminding participant follow-up 
Txt2stop participants enrolled between March 1 and 

June 1, 2009 
Consent was implicit in Txt2stop by choosing to 

provide follow-up or not 
Available at any time withdrawing from Txt2stop 
Minimization 

Completed follow up at 30 weeks from randomization 

Completed follow up at 26 weeks from randomization 
RR for response and 95% CI. Test of homogeneity to 

evaluate a potential effect modification of being 

allocated to Txt2stop intervention or control group 
1 900 participants to detect a difference in follow-up at 

30 weeks of 85% vs 80%; 80% power and 0.05 

significance level (two-sided) 



Single-blind randomized control trial 
Telephone call from senior female clinician and 
researcher inviting participant to complete follow-up 

Standard Txt2stop procedures 

Txt2stop participants > 6 weeks overdue for cotinine 

sample follow-up 
Consent was implicit in Txt2stop by choosing to 

provide follow-up or not 
Available at any time withdrawing from Txt2stop 
Randomization 

Completed cotinine sample follow-up for the Txt2stop 
trial 

No secondary outcome 

RR for response and 95% CI 



127 participants to detect a difference in follow-up at 
35 weeks of 55% vs 30%; 80% power and 0.05 
significance level (two-sided) 



RR: risk ratio, CI: confidence interval. 
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Assessed for eligibility (n=1950) 



Excluded (n=0) 



Allocated through minimisation (n=1950) 



Allocated to control (n=974) 



Allocated to intervention (n=976) 



Analysed (n=974) 
Excluded from analysis (n=0) 



Analysed (n=976) 
Excluded from analysis (n=0) 



Figure 1 Trial 1 flow chart (a refrigerator magnet and a text message to increase participant's retention in Txt2stop) 



The statistical analysis compared the proportions 
of those not lost to Txt2stop follow-up in interven- 
tion and control group using chi-square tests. A risk 
ratio (RR) for response and a 95% CI were calculated. 
A test of homogeneity also was used to evaluate 
potential effect modification of being allocated to 
Txt2stop intervention or control group. 

The study was powered for the primary outcome 
measure. For a difference in follow-up at 30 weeks 
of 85% versus 80%, there was an 80% chance 
(power) that a trial with 1900 subjects, 950 per 
group, would detect the difference with two-sided 
probability <0.05. 

Trial 2 was a RCT of one telephone call from the 
study principal investigator to increase participant 
follow-up. Trial 2 also was a single-blind controlled 
trial, with those recording and assessing outcomes 
blind to the intervention. 

In addition to the standard Txt2stop procedures, 
the intervention group received a telephone call 
from the study principal investigator (female senior 
clinician and researcher), who invited participants 
who were at least 6 weeks overdue in providing a 
cotinine sample to complete follow-up. The control 
group received the standard Txt2stop procedures 
(see Figure 2). 

All Txt2stop participants more than 6 weeks 
overdue for cotinine sample follow-up in April 2009 
were eligible for Trial 2 [12]. As in Trial 1, partic- 
ipants had consented to join the Txt2stop trial and 
any participant withdrawing from the Txt2stop 
trial was also withdrawn from Trial 2. Allocation to 



Trial 2 intervention or control group was performed 
through computer-generated randomization and 
was concealed from the investigators. The study 
principal investigator contacted participants by 
telephone in April 2009. 

The primary outcome of the study was com- 
pleted cotinine sample follow-up at the end of May 
2009 for the Txt2stop trial. 

The statistical analysis compared the proportions 
of those not lost to follow-up in intervention and 
control group using chi-square tests. A RR for 
response and a 95% CI were calculated. 

The study was powered for the primary outcome 
measure. For a difference in follow-up of 55% 
versus 30%, there was an 80% chance (power) 
that a trial with 127 subjects, 65 in intervention 
and 62 in control group, would detect the differ- 
ence with two-sided probability <0.05. 



Results 

Trial 1: 1950 participants were included in the trial. 
As shown in Table 2, the baseline characteristics of 
the Trial 1 population were similar to those of the 
Txt2stop population. Twenty six weeks after ran- 
domization, 33.5% (327 of 976) of participants in 
the intervention group posted the questionnaire 
back, compared with 33.8% (329 of 974) of partic- 
ipants in the control group. The RR for response to 
the refrigerator magnet was 0.99 (95% CI 0.88 to 
1.12). The effect modification by allocation arm 
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Table 2 Demographic characteristics of Txt2stop, Trial 1 and Trial 2 populations 



Variable 


Txt2stop population N (%) 


Trial 1 population N (%) 


Irial 2 popi 


Gender 








V VUI I lei 1 


?60s (AA Q1 


RR3 C45 31 


60 (47.2) 




31 95 (55.1 ) 


1 067 f 71 


67 (52.8) 










< 30 


1572 (27.1) 


573 (29.4) 


64 (50.4) 


30-45 


971 6 (Af\ R1 


R97 C46 (11 


50 (39.4) 


> 45 


1 51 7 f?6 11 

IJ 1 1 y Z_ . 1 1 


480 C74 61 


1 3 (1 0.2) 


Fth n \c itv 

1 LI II 1 1 <_l Ly 








White 


51 36 C88 51 


1 771 (90 81 

1 / / I \^y\J . (J J 


1 09 ("85 81 


Black 


240 (4.1) 


6? n 71 


6 (A 71 




253 (4.4) 


61 C3 11 


R (f, 31 


Other 


1 34 O 31 


45 (7 31 


3 (7 41 


1 Inknnwn 

\J 1 MM lUvv 1 1 


37 (0.6) 


1 1 (0.6) 


1 CO 81 


Age left school 








<16 


2538 (43.8) 


985 (50.5) 


54 (42.5) 


>16 


3262 (56.2) 


965 (49.5) 


73 (57.5) 


Employment 








Manual 


1789 (30.8) 


523 (26.8) 


28 (22.0) 


Nonmanual 


2539 (43.8) 


679 (34.8) 


45 (35.4) 


N/A unknown 


1472 (25.4) 


748 (38.4) 


54 (42.5) 


Fagerstrom index 








<5 


3488 (60.1) 


1154 (59.2) 


59 (46.5) 


>5 


2312 (39.9) 


796 (40.8) 


68 (53.5) 



within the Txt2stop intervention and control 
groups was tested; the test of homogeneity 
showed no interaction (p-value = 0.92). 

After 30 weeks of randomization, 83.3% (813 of 
976) of participants in the intervention group 
posted the questionnaire back, compared with 
82.2% (801 of 974) of participants in the control 
group. The RR for response was 1.01 (95% CI 0.97 
to 1.05). Again, no effect modification by allocation 
arm within the Txt2stop trial was detected (p-value 
from the test of homogeneity = 0.83). 

Trial 2: A total of 127 participants were included 
in the analysis. As shown in Table 2, the baseline 
characteristics of the population in Trial 2 were 
similar to those of the population in Txt2stop in 
regard to sex and ethnicity. The Trial 2 population 
was younger and slightly more educated than the 
total Txt2stop population; a lower proportion were 
manual workers and fewer Trial 2 participants had 
serious nicotine dependence. 

In all, 31% (20 of 65) of participants in the 
intervention group completed follow-up by send- 
ing cotinine samples, compared with 32% (20 of 
62) in the control group. The RR was 0.93 (95% CI 
0.44 to 1.98). 



Discussion 

Information emphasizing the benefits to society, 
via written information on a refrigerator magnet 



and a subsequent text message, did not increase the 
return of completed questionnaires at 26 weeks or 
30 weeks after randomization. A single telephone 
call from a female senior clinician and researcher 
did not increase return of mailed saliva samples. 
These interventions had no additional effect on 
follow-up when evaluated within a trial, where all 
other interventions known to increase follow-up 
[5,10] already had been implemented. These trials 
were pragmatic and, apart from the interventions 
tested, participants received all interventions to 
increase follow-up that were part of the Txt2stop 
standard operating procedures. 

The impact of the intervention emphasizing the 
benefits to society of completing follow-up may 
have been reduced, since follow-up letters in the 
standard Txt2stop follow-up procedures already 
emphasized these benefits to some extent. The 
Txt2stop standard procedures included telephone 
calls from trial assistants, who could be male or 
female. The senior female clinician and researcher 
was not the only female member of staff calling 
Txt2stop participants. 

For Trial 2, we included the entire sample of 
participants available at the time. With this sample, 
Trial 2 had sufficient power to detect an absolute 
difference of 25% between intervention and con- 
trol groups. Ideally, trials should be designed 
to detect 'meaningful' differences in the outcome. 
Even modest increases in follow-up could 
reduce bias. Depending on the existing level of 
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Assessed for eligibility (n=127) 



Excluded (n=0) 



Allocated through randomisation (n=127) 



Allocated to control (n=62) 



Analysed (n=62) 
Excluded from analysis (n=0) 



f 


Allocated to intervention (n= 


=65) 




K 








j 






r 


Analysed (n=65) 
Excluded from analysis (n= 


=0) 


\ 


\ 








J 



Figure 2 Trial 2 flow chart (a telephone call from the study principal investigator requesting follow-up in Txt2stop) 



trial follow-up, the number of trial participants, 
and the alternate priorities for a senior investiga- 
tor's time, it could be worthwhile for a senior 
investigator to put in the time to increase follow-up 
by less than 25% (for example, a 5-10% absolute 
increase in follow-up could be important). A lim- 
itation of Trial 2 was that the trial was underpow- 
ered to detect an absolute increase in follow-up of 
less than 25%. 

The Trial 1 control group received a text message 
very similar to the text sent to the Trial 1 interven- 
tion group but without any comment emphasizing 
benefits to society. This text message was sent 
because telephone text messages have been shown 
to work as effective reminders for appointments 
and medications reminders [15,16]. Several days of 
strike of the Royal Mail service during 3 weeks in 
October and November 2009 [17] seriously affected 
the return of Txt2stop questionnaire. The strike is 
likely to have decreased the return of question- 
naires in the early phase. During the strike, it is 
possible that some returned questionnaires may 
have been lost by Royal Mail and attempts to obtain 
the data for a second time may have been less 
effective than the first request. There is no reason to 
expect that the strike would have differentially 
affected follow-up between the intervention and 
control group, and so the strike may have affected 
the precision of the result but is unlikely to have 
influenced the direction of effect. 

We found no evidence that either emphasizing 
the benefits to society of trial participation to 



participants or a call from a female senior clinician 
and researcher influence follow-up, when other 
evidence-based methods to decrease loss to follow- 
up are already employed. 
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